Apparatus and method for determining cosmetic skin attributes

ABSTRACT

A method of determining a cosmetic skin attribute of a person is provided. The method includes obtaining a color channel image of a person&#39;s skin, analyzing the color channel image with a computer using entropy statistics to obtain an entropy value, and then determining a cosmetic skin attribute for the person based on the entropy value.

TECHNICAL FIELD

The present invention relates to an apparatus and method for determiningcosmetic skin attributes.

BACKGROUND

Skin imaging methods have been widely used to study different phenotypesof skin aging. Numerous image analysis techniques and algorithms havebeen proposed in literature to characterize aging skin particularlyfocusing on aging phenotypes such as wrinkles, spots and sagging. It isknown that appearance of skin aging related phenotypes is a continuousprocess over time. For example, uneven pigmentation may first causevisually imperceivable spots, which eventually may become visible overtime. As such, a younger consumer (e.g., less than 30 years of age)generally does not have classic visible skin aging related phenotypesand therefore has the impression that there are no pressing needs toprevent, delay, and/or mitigate such visibly imperceivable phenotypes ofaging, until it is too late.

U.S. Publication Number 2010/0284610A1 (“the '610 Publication”) relatesto a skin color evaluation method for evaluating skin color from aninput image including a face region. The '610 Publication describesdividing a face region of the image into predetermined regions accordingto first feature points formed of at least 25 areas that are setbeforehand and second feature points that are set by using the firstfeature points. The '610 Publication further describes performing a skincolor distribution evaluation by generating a skin color distributionbased on average values using at least one of L*, a*, b*, C_(ab)*, andh_(ab) of a L*a*b* color system, tri-stimulus values X, Y, Z of an XYZcolor system and the values of RGB, hue H, lightness V, chroma C,melanin amount, and hemoglobin amount, followed by performing evaluationbased on measured results with respect to the regions that are dividedand displaying the measured results or evaluation results on a screen.However, the '610 Publication is focusing on visible skin colordistribution and it fails to describe skin aging related phenotypes, andtherefore does not describe any method for evaluating visuallyimperceivable skin aging related phenotypes.

Accordingly, there is a need for a method for determining cosmetic skinattributes of a person so as to enable pro-active skincare treatment atan earlier stage.

SUMMARY OF THE INVENTION

The present invention relates to a method of determining a cosmetic skinattribute of a person, the method comprising the steps of:

-   -   a) obtaining at least one color channel image comprising at        least one portion of skin of the person;    -   b) analyzing the at least one color channel image using entropy        statistics to obtain an entropy value; and    -   c) determining the cosmetic skin attribute of the at least one        portion of skin of the person based on the entropy value.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating an exemplary system for determining acosmetic skin attribute over a network according to the presentinvention;

FIG. 2 is a diagram illustrating an alternative exemplary system fordetermining a cosmetic skin attribute according to the presentinvention;

FIG. 3 is a block diagram illustrating components of an exemplaryapparatus for determining a cosmetic skin attribute according to thepresent invention;

FIG. 4 is a block diagram illustrating a method for determining acosmetic skin attribute according to the present invention;

FIG. 5 is a flow chart illustrating a method for determining a methodfor determining a cosmetic skin attribute according to the presentinvention;

FIGS. 6A to 6C are a series of process flow diagrams illustrating amethod for determining a cosmetic skin attribute according to thepresent invention;

FIG. 7 is a flow chart illustrating a method for determining a cosmeticskin attribute according to the present invention;

FIGS. 8A and 8B are process flow diagrams illustrating a method ofvisualizing entropy values of at least one cosmetic skin attribute of atleast a portion of a face of a subject in a digital image according tothe present invention;

FIG. 9 is a flow chart illustrating a method of visualizing entropyvalues of at least one cosmetic skin attribute of at least a portion ofa face of a subject in a digital image according to the presentinvention;

FIGS. 10A to 10C are a series of process flow diagrams illustratingdetails of a step of obtaining a first digital image in a method ofdetermining a cosmetic skin attribute according to the presentinvention;

FIG. 11 is a flow chart illustrating the steps of obtaining the firstdigital image;

FIG. 12 is a picture illustrating a step of defining a plurality oftiles in in a method of determining a cosmetic skin attribute accordingto the present invention;

FIG. 13 is a flow chart illustrating the steps of defining the pluralityof tiles;

FIGS. 14A to 14C are process flow diagrams illustrating a process ofdisplaying the plurality of tiles according to the present invention;

FIG. 15 is a flow chart illustrating a process of displaying theplurality of tiles according to the present invention;

FIGS. 16A to 16D are process flow diagrams illustrating a method ofvisualizing entropy values of at least one cosmetic skin attributeaccording to the present invention;

FIG. 17 is a flow chart illustrating a method of visualizing entropyvalues of at least one cosmetic skin attribute according to the presentinvention;

FIG. 18 is a flow chart illustrating an alternate method of visualizingentropy values of at least one cosmetic skin attribute according to thepresent invention;

FIGS. 19A to 19E are screen shots, each illustrating an exemplary userinterface for visualizing entropy values of at least one cosmetic skinattribute according to the present invention;

FIG. 20 is a screen shot illustrating an exemplary user interfacecomprising a heat map as an example of an image description forvisualizing at least one cosmetic skin attribute according to thepresent invention;

FIG. 21 is a screen shot illustrating an alternate variation of an imagedescription for visualizing at least one cosmetic skin attribute in theuser interface of FIG. 19; and

FIG. 22 is a screen shot illustrating an exemplary user interface forvisualizing at least one cosmetic skin attribute according to thepresent invention.

DETAILED DESCRIPTION

It is known that when the skin is subjected to stress (caused by UV,aging, mental, environmental factors), skin will be damaged at variouslevels including DNA level, cellular level and tissue level. Suchdamages to the skin can result in skin imperfections. Presence of theseskin imperfections significantly impact on optics of the skin such asdescribed in the following examples:

-   -   If the stratum corneum is dry (winter/air condition), light will        reflect more from the surface (surface reflection) enhancing        skin micro texture which cause lines on the skin    -   If the dermis is damaged (UV), less light will scatter at dermis        (also known as dermis-scattering) and light will penetrate skin        (less subsurface reflection). When there is less        demis-scattering, and the skin appears darker as a result.    -   If skin is exposed to chronic UV, skin produces more melanin.        Melanin absorbs light reducing subsurface reflection and hence        skin appears darker.

The above skin imperfections manifest as visually imperceivable signalsof poor skin quality to the consumers eye. Consumers may consider theseimperfections as impurities at a subconscious level, but are not able totake action to improve the imperfections because of lack of consciousknowledge. If the skin is subject to chronic stress and/or untreated,these visually imperceivable impurities may eventually lead to visibleand perceivable phenotypes (pigmented spots, wrinkles, sagging).

The present invention relates to a method, apparatus and system fordetermining at least one cosmetic skin attribute of a subject and agraphical user interface. The method comprises the steps of (a)obtaining at least one color channel image comprising at least oneportion of skin of the person; (b) analyzing the at least one colorchannel image using entropy statistics to obtain an entropy value; and(c) determining the cosmetic skin attribute of the at least one portionof skin of the person based on the entropy value.

It has been surprisingly found that entropy values obtained by analyzinga color channel image comprising at least one portion of skin of aperson can be used determine cosmetic skin attributes that are visuallyperceivable as well as visually imperceivable cosmetic skin attributes.

As explained below, if a cosmetic skin attribute of a person can bedetermined based on an entropy value that is obtained by analyzing acolor channel image comprising at least one portion of skin of a personusing entropy statistics, useful information can be inferred (e.g.condition of the cosmetic skin attribute) so that the person can seekproactive skincare treatment to improve the condition of the cosmeticskin attribute. Specifically, as described hereinafter, a color channelimage having at least one portion of skin of a person may have a regionof interest on the at least one portion of skin that is having a lowerentropy value relative to other regions of interest. A lower entropyvalue in the region of interest shows less intensity variation which isindicative of a better cosmetic skin attribute condition. Accordingly, ahigher entropy value in other regions of interest show more intensityvariation which is indicative of a poorer cosmetic skin attributecondition. A further advantage is that the proactive skincare treatmentcan be targeted to a specific region of interest.

Prior to describing the present invention in detail, the following termsare defined and terms not defined should be given their ordinary meaningas understood by a skilled person in the relevant art.

“Entropy” as used herein refers to a Shannon entropy (E) of a discreterandom distribution (p(x)) and is defined by the following equation:

$\begin{matrix}\left. {{E(p)} = {- {\sum\limits_{x}{{p(x)} \times \log\;{p(x)}}}}} \right) & (1)\end{matrix}$

wherein p(x) is the distribution of grey levels

E(p) represents the amount of information in a digital image or a colorchannel image in a color system after conversion of the digital image tothe color channel image. “Entropy statistics” as used herein refers to astatistical method that uses entropy as a descriptive statistic foranalyzing digital images or color channel images. In a non-limitingexample wherein the digital image is an RGB image, entropies (entropyvalues) for each R (red), G(green) and B(blue) channel can be calculatedseparately. The entropy value of an image can be calculated bycalculating at each pixel position (i,j) the entropy value of thepixel-values within a 2-dimensional region centered at (i,j). The2-dimensional region may be a part of a color channel image. Programmingsoftware packages such as Python may be used to calculate the entropyvalue.

“Cosmetic skin attribute” as used herein includes all skin attributesthat provide a visual/aesthetic effect on an area of the human body orimpact skin appearance and/or feel. Some non-limiting examples of acosmetic skin attribute may include skin purity, skin age, skintopography, skin tone, skin pigmentation, skin pores, skin inflammation,skin hydration, skin sebum level, acne, moles, skin radiance, skinshine, skin dullness, uneven tone, or skin barrier. It will beappreciated by a skilled person that the above cosmetic skin attributesare standard terms, and a corresponding definition of the cosmetic skinattribute may be found in the following published references namely,“Handbook of cosmetic science and technology, 3^(rd) edition, editorsAndre O. Barel, Marc Pave, Howard I. Maiback, CRC Press, 2009”,“Cosmetic Science and Technology-Theoretical Principles andApplications, editors Kazutami Sakamoto Robert Y. Lochhead, Howard I.Maibach, Yuji Yamashita, Elsavier, 2017”, “Cosmetic Dermatology:Products and Procedures, Editor(s): Zoe Diana Draelos, BlackwellPublishing Ltd, 2010”. Cosmetic skin attributes do not include skinattributes related to medical conditions or underlying medicalconditions.

“Imperceivable cosmetic skin attribute” as used herein refers to acosmetic skin attribute that cannot be perceived or is imperceptible bythe perceiver, i.e. a person, a user, or a human subject. Perceivederives from the word “Perception” which refers to the organization,identification, and interpretation of sensory information in order torepresent and understand the presented information, or the environment.All perception involves signals that go through the nervous system,which in turn result from physical or chemical stimulation of thesensory system. For example, vision involves light striking the retinaof the eye, smell is mediated by odor molecules, and hearing involvespressure waves. Perception is not only the passive receipt of thesesignals, but it is also shaped by the recipient's learning, memory,expectation, and attention. Perception can be split into two processes,i.e. process (1) that relates to processing the sensory, input, whichtransforms these low-level information to higher-level information(e.g., extracts shapes for object recognition), and process (2) thatrelates processing which is connected with a person's concepts andexpectations (or knowledge), restorative and selective mechanisms (suchas attention) that influence perception. For example, a perceiver maysee an object in process (1) but does not have the knowledge to perceiveand recognize what the object represents/mean in process (2), andtherefore may regard the object to be visually imperceivable.

“Visually imperceivable cosmetic skin attribute” as used herein includesall cosmetic skin attributes which are not detectable by an unaided eyeor a cosmetic skin attribute detectable visually by a consumer but theconsumer does not understand the cosmetic skin attribute, and thereforeregarded as imperceivable cosmetic skin attributes. Some nonlimitingexamples of a visually imperceivable cosmetic skin attribute that is notdetectable visually by the unaided eye include cosmetic skininflammation, skin sebum level, or any underlying cosmetic skinattribute.

“Unaided” as used herein means without assistance from diagnosticequipment.

“Tile” as used herein includes a unit, such as for example a pixel, thatform a part of a digital image and accordingly “Tiles” form the whole ofthe digital image.

“Digital image data” as used herein includes image data obtained from animage obtaining device including but not limited to a digital camera, aphoto scanner, a computer readable storage medium capable of storingdigital images, and any electronic device including picture takingcapabilities. Digital image data may also include color channel imageswhich are converted from a RGB image into a color channel image in acolor system.

“Single degree of indicium” as used herein includes all electronicvisual representations including but not limited to a graphical symbol,a numerical value, a color code, illumination techniques andcombinations thereof.

“Skin Attribute Index” as used herein refers to a score that can becalculated based on a mathematical formula or a model derived fromstatistical methods and data or a lookup table (an array ofinformation). The Skin Attribute Index may be generated as a probabilityvalue indicative of a condition of the cosmetic skin attribute of the atleast one portion of skin of the person relative to a defined populationof people, preferably the Skin Attribute Index is generated as afunction of the entropy value defined by F(Entropy Value), wherein saidfunction is determined by a model established upon a training datasetwherein the training dataset comprises: (i) a plurality of color channelimages of a the defined population of people, wherein each of theplurality of color channel images comprises facial skin of a person inthe defined population of people, wherein the facial skin comprises thecosmetic skin attribute; (ii) an associated class definition based onthe cosmetic skin attribute.

“L*a*b*” as used herein, refers to the commonly recognized color spacespecified by the International Commission on Illumination (“CIE”). Thethree coordinates represent (i) the lightness of the color (i.e., L*=0yields black and L*=100 indicates diffuse white), (ii) the position ofthe color between magenta and green (i.e. negative a*values indicategreen while positive a*values indicate magenta) and (iii) the positionof the color between yellow and blue (i.e. negative b*values indicateblue and positive b*values indicate yellow).

“Chromophore mapping” as used herein, refers to the commonly recognizedchromophore space for melanin and hemoglobin mapping and determiningmelanin or hemoglobin concentration which may be used as an indicator ofoverall skin tone. Mean melanin or hemoglobin may be calculated from thechromophore map data. Additionally, skin tone evenness can be determinedby melanin or hemoglobin evenness (e.g. standard deviation) which alsomay be calculated from the chromophore map data.

“Skin purity” as used herein, appearance of the absence of skinimperfections in at least of portion of skin of a person. The skinimperfections include cosmetic skin attributes which impact irregular ornon-uniform spectral properties composed of the surface reflection ofthe skin topographical morphology and/or the sub-surface reflection ofskin chromophores such as melanin, haemoglobin and/or keratinocyte andfibroblast oriented cellular metabolites, and include but are notlimited to skin radiance, skin tone or the like.

“Skin age” as used herein, means apparent age which refers to the age ofskin of a person that is visually estimated or perceived to be, comparedto norm age skin appearances, based on the physical appearances,preferably a face of the person, preferably at least a portion of a faceof the person, more preferably, at least one region of interest (ROI) ofthe at least a portion of a face of the person, even more preferably,the at least one ROI is selected from the group consisting of: a skinregion around the eye (“eye region”), a skin region around the cheek(“cheek region”), a skin region around the mouth (“mouth region”), andcombinations thereof.

“Skin tone” as used herein, generally refers to the overall appearanceof basal skin color or color evenness. Skin tone is typicallycharacterized over a larger area of the skin. The area may be more than100 mm2, but larger areas are envisioned such as the entirety of thefacial skin or other bodily skin surfaces (e.g. arms, legs, back, hands,neck).

“Skin wrinkle” as used herein, generally refers to a fold, ridge orcrease in the skin and includes but is not limited to fine lines, superfine lines, fine wrinkles, super fine wrinkles, wrinkles, lines. Skinwrinkle may be measured in terms of, for example, density and/or length.

“Skin radiance” as used herein, generally refers to an amount of lightthat the skin reflects, and may be referred to as skin shine.

“Skin texture” as used herein, generally refers to the topology orroughness of the skin surface.

“Skin tension” as used herein, generally refers to the firmness orelasticity of the skin.

“Skin sebum level” as used herein, generally refers to an amount ofsebum which is an oily or waxy matter secreted by sebaceous glands inthe skin.

“Skin spots” as used herein, generally refers discoloration or unevenpigmentation (e.g hyperpigmentation, blotchiness) of the skin. Skinspots may be evaluated in terms of, e.g. density, size, and/or degree ofdiscoloration.

“Skin care product” as used herein, refers to a product that includes askin care active and regulates and/or improves skin condition.

“Digital image” as used herein, refers to a digital image formed bypixels in an imaging system including but not limited to standard RGB,or the like and under images obtained under different lightingconditions and/or modes. Non-limiting examples of a digital imageinclude color images (RGB), monochrome images, video, multispectralimage, hyperspectral image or the like. Non-limiting light conditionsinclude white light, blue light, UV light, IR light, light in a specificwavelength, such as for example light source emitting lights from 100 to1000 nm, from 300 to 700 nm, from 400 to 700 nm or differentcombinations of the upper and lower limits described above orcombinations of any integer in the ranges listed above. The digitalimage may be obtained from an image obtaining device including but notlimited to a digital camera, a photo scanner, a computer readablestorage medium capable of storing digital images, and any electronicdevice including picture taking capabilities.

In the following description, the method described is a method fordetermining a cosmetic skin attribute. According, the apparatusdescribed is an apparatus for determining a cosmetic skin attribute. Theapparatus may also be configured for generating for display, on adisplay, entropy statistics of digital image data of at least a portionof a face of a subject, and the graphical user interface described is agraphical user interface for displaying entropy statistics of thedigital image data of at least a portion of a face of the subject. Thesystem described is an entropy-based system for determining a cosmeticskin attribute. In an exemplary embodiment, the system is a stand-aloneimaging system (shown in FIG. 2) that is located at a retail cosmeticscounter for the purpose of analyzing and recommending cosmetic and skincare products. However, it is contemplated that the system and themethod may be configured for use anywhere, such as for example as shownin FIG. 1, through an electronic portable device comprising an imageobtaining unit/device and a display, wherein the electronic portabledevice is connected to an apparatus for generating for display on adisplay, a graphical user interface for visualizing entropy value of acosmetic skin attribute through a network.

FIG. 1 is a schematic diagram illustrating a system 10 for visualizing acosmetic skin attribute according to the present invention. The system10 may include a network 100, which may be embodied as a wide areanetwork (such as a mobile telephone network, a public switched telephonenetwork, a satellite network, the internet, etc.), a local area network(such as wireless-fidelity, Wi-Max, ZigBee™, Bluetooth™, etc.), and/orother forms of networking capabilities. Coupled to the network 100 are aportable electronic device 12, and an apparatus 14 for generating fordisplay on a display, a graphical user interface for visualizing acosmetic skin attribute. The apparatus 14 is remotely located andconnected to the portable electronic device through the network 100. Theportable electronic device 12 may be a mobile telephone, a tablet, alaptop, a personal digital assistant and/or other computing deviceconfigured for capturing, storing, and/or transferring a digital imagesuch as a digital photograph. Accordingly, the portable electronicdevice 12 may include an input device 12 a for receiving a user input,an image obtaining device 18 such as a digital camera for obtainingimages and an output device 12 b for displaying the images. The portableelectronic device 12 may also be configured for communicating with othercomputing devices via the network 100. The portable electronic device 12may further comprise an image processing device (not shown) coupled withsaid imaging obtaining device 18 for analyzing the obtained at least onecolor channel image using entropy statistics to obtain an entropy valueand determining the cosmetic skin attribute of the at least one portionof skin of the person based on the entropy value. The image processingdevice preferably comprises a processor with computer-executableinstructions. The portable electronic device 12 may further comprise adisplay generating unit (not shown, such as an electronic LED/LCDdisplay) for generating a display to display content data describing thedetermined cosmetic skin attribute.

The apparatus 14 may include a non-transitory computer readable storagemedium 14 a (hereinafter “storage medium”), which stores image obtaininglogic 144 a, image analysis logic 144 a and graphical user interface(hereinafter “GUI”) logic 144 c. The storage medium 14 a may compriserandom access memory (such as SRAM, DRAM, etc.), read only memory (ROM),registers, and/or other forms of computing storage hardware. The imageobtaining logic 144 a, image analysis logic 144 b and the GUI logic 144c define computer executable instructions. A processor 14 b is coupledto the storage medium 14 a, wherein the processor 14 b is configured to,based on the computer executable instructions, for implementing a method90 for determining a cosmetic skin attribute of a subject according tothe present invention as described herein after with respect to theblock diagram of FIG. 4 and the flowchart of FIG. 5. The cosmetic skinattribute may be a visually imperceivable cosmetic skin attribute,wherein the visually imperceivable cosmetic skin attribute is a cosmeticskin attribute which is not detectable by an unaided eye, or a cosmeticskin attribute detectable visually by a consumer but the consumer doesnot understand the cosmetic skin attribute. An advantage of determiningvisually imperceivable cosmetic skin attributes is to enable consumersto make informed decisions and take pro-active action to improve thecondition of the visually imperceivable cosmetic skin attributes.

Determination Method

Referring to FIGS. 4 and 5, when the processor 14 b is initiated, theprocessor 14 b causes at least one color channel image 60L comprising atleast one portion of skin of the person to be obtained in step 91, e.g.via conversion of a digital image into a color channel image in a colorsystem which will be described hereinafter with reference to FIGS. 6A,6B, 6C and 7. The at least one color channel image 60L is analyzed instep 92 using entropy statistics to obtain an analysis output 80 whereinthe analysis output 80 comprises an entropy value. In step 93, thecosmetic skin attribute of the at least one portion of skin of theperson is determined based on the entropy value.

At least one color channel image may be an image in a color systemselected from the group consisting of L*a*b* color system, RGB colorsystem, HSL/HSV color system, and CMYK color system.

Table 1 below sets out each entropy value with a corresponding colorchannel image and corresponding cosmetic skin attributes to bedetermined based on the entropy value. The color channel image describedin Table 1 is an image in the L*a*b* color system selected from thegroup consisting of a L channel image, an a-channel image, a b-channelimage, a c-channel image, and combinations thereof.

TABLE 1 Color Channel Image Entropy Value Cosmetic Skin Attribute Lchannel image L-entropy value skin purity, skin tone, skin radiancea-channel image a-entropy value skin inflammation b-channel imageb-entropy value skin pigmentation or skin dullness c-channel imagec-entropy value Skin topography, including but not limited to pores,wrinkles, fine lines, sagging, skin elasticity and combinations thereof.

Determining the cosmetic skin attribute may comprise generating a SkinAttribute Index as a probability value indicative of a condition of thecosmetic skin attribute of the at least one portion of skin of theperson relative to a defined population of people. Specifically, in avisual perception study, consumers may be asked to rank digital images(e.g. photographs) of the defined population of people for a cosmeticskin attribute based on a predetermined scale. The ranked digital imagesmay be stored as a database so as to be analyzed according to the method90 to determine an entropy value that has the highest correlation withthe cosmetic skin attribute.

Alternatively, the Skin Attribute Index may be generated as a functionof the entropy value defined by a function, F(Entropy Value), whereinsaid function is determined by a model established upon a trainingdataset. The training dataset may comprise: (i) a plurality of colorchannel images of a defined population of people, wherein each of theplurality of color channel images comprises facial skin of a person inthe defined population of people, wherein the facial skin comprises thecosmetic skin attribute; (ii) an associated class definition based onthe cosmetic skin attribute. Techniques for building training datasetsare known to a person skilled in the field of image processing methodsand will not be further described.

The model may be a regression model or a classification model,preferably a linear regression model, more preferably a machine learninglinear regression model, most preferably a machine learning supportvector regression (SVR) model. The SVR model is a specific example of aSupport Vector Machine (SVM) model. A machine learning model may also bea support vector classification model or a random forest regressionmodel.

Using a SVR model enables the advantages of accuracy, reproducibility,speed in the performance of the method when implemented as a nativeapplication on a portable electronic device. In particular, the weightof a SVR model allows the native application to have a smaller hardwarefootprint, and consequently the methods according to the presentinvention may be easily deployed in portable electronic devices such asmobile phones with mobile phone operating systems (OS) including but notlimited to iOS for the Apple™ phone or Android OS for Android phones.

The classification model may be used to classify consumers into aplurality of groups, each group having different degrees of a conditionof the same cosmetic skin attribute, preferably two groups, morepreferably three groups so as to define an associated class definitionbased on the numerical value of the Skin Attribute Index. For example,the method may display a heat map configured to classify regions of theskin into a high level of a cosmetic skin attribute condition or a lowlevel of a cosmetic skin attribute condition based on thresholdsassigned to each of the groups.

The at least one color channel image is an image in the L*a*b* colorsystem selected from the group consisting of a L color channel image, ana-channel image, a b-channel image and a c-channel image from RGB colorsystem, and combinations thereof; wherein the entropy value is selectedfrom the group consisting of a L-entropy value, an a-entropy value, ab-entropy value, a c-entropy value, and combinations thereof; andwherein the function has the following formula:Skin Attribute Index=A+B×(L-entropy value)+C×(a-entropyvalue)+D×(b-entropy value)+E×(c-entropy),

wherein A, B, C, D, and E are constants; wherein at least one of B, C,D, and E is not 0.

It will be appreciated that the constants A, B, C, D, and E may varybased on the size and content of the training dataset, and may be anynumerical value generated by the model based on the training dataset.

Specifically, each one of the entropy values above may be used alone orin combination with another one of the entropy values. For example usinga single entropy value may result in faster computing speed whichenables small devices with very basic hardware to be used, therebyresulting in a more efficient and cost effective product.

The at least one color channel image may be a L channel image; whereinthe entropy value is a L-entropy value; wherein C, D, E each has a valueof 0; and wherein the generated Skin Attribute Index is indicative ofskin purity, skin tone or skin radiance.

It is known that when the skin is subjected to stress (caused by UV,aging, mental, environmental factors), skin will be damaged at variouslevels including DNA level, cellular level and tissue level. Suchdamages to the skin can result in skin imperfections. Presence of theseskin imperfections significantly impact on optics of the skin such asdescribed in the following examples:

-   -   If the stratum corneum is dry (Winter/Air condition), light will        reflect more from the surface (surface reflection) enhancing        skin micro texture which cause lines on the skin    -   If the dermis is damaged (UV), less light will scatter at dermis        (also known as dermis-scattering) and light will penetrate skin        (less subsurface reflection). When there is less        demis-scattering and hence skin appear darker.    -   If skin is exposed to chronic UV, skin produces more melanin.        Melanin absorbs light reducing subsurface reflection and hence        skin appear darker.

The above skin imperfections manifest as imperceivable signals of skinquality to the consumers eye. Consumers may consider these imperfectionsas impurities, but are not able to take action to improve theimperfections because of lack of knowledge. If the skin is subject tochronic stress and/or untreated, these imperceivable impurities mayeventually lead to visible and perceivable phenotypes (pigmented spots,wrinkles, sagging).

It has been surprisingly found that a L-entropy value of a L colorchannel image has the highest correlation to skin purity.

A technical effect of selecting L-channel image as the at least onecolor channel image for a analyzing step to obtain a L-entropy value andto determine skin purity based on the L-entropy value according tomethods according to the present invention is because L-entropy valuehas the highest correlation (r=0.89) to skin purity relative to otherentropy values based on analyzing the color channel images. Below isdata generated based on correlation with results from a visualperception study using statistical analysis using Pearson correlationcoefficient (r). The correlation results are shown below in Table 2below.

TABLE 2 Pearson Correlation Coefficient (r) with Entropy Value resultsof Visual Perception Study L-entropy value 0.89 a-entropy value 0.55b-entropy value 0.7 c-entropy value 0.76

A higher Pearson correlation coefficient (r) means that the entropyvalue is a factor that contributes more to the condition of the cosmeticskin attribute that is studied in the visual perception study.Specifically, the visual perception study is conducted based on apredetermined number of panelists=302, age of the panelists=20-50. Thepanelists are asked rank photographs of people for skin purity (as anexample of the cosmetic skin attribute) on a scale of 1 to 5. Based onthe visual perception study results and above correlation results, ithas been found that L channel entropy value of the filtered image (byfrequency filter) has the highest correlation with the skin purityattribute. Therefore, use of the L-entropy value of the L channel todetermine skin purity of at least a portion of skin of a person in adigital image can be used to transform skin purity from a visuallyimperceivable cosmetic skin attribute into an explainable cosmetic skinattribute in a consumer relevant way to consumers.

The at least one color channel image may be an a-channel image; whereinthe entropy value is an a-entropy value; wherein B, D, E each has avalue of 0; and wherein the generated Skin Attribute Index is indicativeof skin inflammation.

The at least one color channel image may be a b-channel image; whereinthe entropy value is a b-entropy value; wherein B, C, E each has a valueof 0; and wherein the generated Skin Attribute Index is indicative ofskin pigmentation or skin dullness.

The at least one color channel image may be a c-channel image; whereinthe entropy value is a c-entropy value; wherein B, C, D each has a valueof 0; and wherein the generated Skin Attribute Index is indicative ofskin topography, which is preferably selected from the group consistingof: pores, fine lines, wrinkles, sagging, skin elasticity, andcombinations thereof.

Obtaining Color Channel Image

The color channel image 60 a, 60 b, 60 c may be obtained from a digitalimage 51 as described hereinafter with reference to FIGS. 5A, 5B, 5C and6. Referring to FIG. 1, the network 100 may be used to acquire digitalimages from the portable electronic device 12 and transmitting thedigital images to the apparatus 14 to be used in a method 200 fordetermining a cosmetic skin attribute according to the presentinvention. The input device 12 a may be coupled to or integral with theportable electronic device 12 for receiving a user input for initiatingthe processor 14 b. The portable electronic device 12 may comprise anoutput device 12 b for displaying the plurality of tiles, each havinguniquely assigned single degree of indicium. The input device 12 a mayinclude but is not limited to a mouse, a touch screen display, or thelike.

Referring to FIGS. 1, 5A and 6, when the processor 14 b is initiated,the processor 14 b causes a digital image 51 of at least a portion of aface of the subject to be obtained, e.g. via image obtaining logic 144 ain step 202. The obtained digital image 51 may be a RGB XP digital imageor a RGB shine digital image. The digital image 51 in RGB system isconverted from an RGB image to a digital image data, such as a colorchannel image in a different color system. The processor 14 b furthercauses at least one color channel image 60 a, 60 b, 60 c to beextracted, e.g. via image analysis logic 144 b, from the obtaineddigital image 51 in step 204. The at least one color channel may be acolor channel image that is obtained by further processing of RGB colorchannels based on an equation, for example, 0.1R+0.2G+0.7B. The at leastone color channel image 60 a, 60 b, 60 c may be selected from any one ofcolor channels 60 a, 60 b, 60 c in a color system. In step 206, theextracted at least one color channel image 60 a, 60 b, 60 c is filteredusing a frequency filter. The filtered at least one color channel image61 a, 61 b, 61 c is analyzed using entropy statistics in step 208 todetermine the cosmetic skin attribute of the person. Use of a frequencyfilter in step 206 removes noise from the extracted at least one colorchannel image 60 a, which increases sensitivity of the analysis in step208, thereby resulting in higher accuracy in an analysis output fromstep 208 relative to analyzing a non-filtered color channel image.However, analyzing a non-filtered color channel image may beadvantageous to reduce usage in computing hardware, such as reducinghardware footprint, data storage space or processing capability in theevent that only very minimal and basic hardware is available forimplementing the methods according to the present invention.

Optionally, the method 200 may further comprise applying an imagecorrection factor to the filtered color channel for optimizingparameters in the filtered color channel prior to analyzing the filteredcolor channel. The parameters may include illumination correction, blurcorrection, rotation correction or the like.

A technical effect of determining at least one skin attribute using themethod 90, 200 according to the present invention is that it providesquick and accurate analysis of the cosmetic skin attributes. Table 3below describes an age correlation with commonly used imaging end pointsand the correlation for entropy statistics have a better correlation(0.81) relative to the other imaging end points. Imaging end points mayalso be described as imaging methods for analyzing skin features.

TABLE 3 Endpoint Correlation Texture area fraction 0.43 Spot areafraction 0.55 Wrinkle area fraction 0.57 L -0.52 a 0.39 b 0.54L-Standard deviation 0.02 a-Standard deviation 0.28 b-Standard deviation0.35 Haralick Contrast 0.65 Entropy Statistics 0.81

The method 90, 200 may be performed in less than 5 seconds, preferablyfrom 0.01 second to 5 seconds, more preferably from 0.5 seconds to 1second, or different combinations of the upper and lower limitsdescribed above or combinations of any integer in the ranges listedabove. As the method 90, 200 can be performed in less than 1 second, themethod 200 may be implemented in commercially available hardware such asa portable handheld electronic device including but not limited to amobile phone which is commercially advantageous because of itsscalability to a wider network of consumers.

The color system may be an L*a*b color system and the at least one colorchannel image may be a red color channel 60 a, a yellow color channel 60b or a blue color channel 60 c corresponding a texture channel as shownin FIG. 5B. FIG. 5C shows the filtered color channel images 61 a, 61 b,61 c, each filtered color channel image is analyzed to obtain entropyvalues describing the analyzed color channel image. An entropy value ofthe filtered red color channel 61 a may be defined as a-entropy, anentropy value of the filtered yellow color channel 61 b may be definedas b-entropy and an entropy value of the filtered blue color channel 61c may be defined as c-entropy.

The color system may be a chromophore mapping space as describedhereinbefore and the at least one color channel image may be ahemoglobin channel image or a melanin channel image.

The frequency filter may be a Fast Fourier transformation filter, aWavelet transformation filter or a Difference of Gaussian (DoG) filter.More preferably, the frequency filter is a DoG filter. The DoG filterhas a Gaussian filter 1 and a Gaussian filter 2. The Gaussian filter 1may comprise a standard deviation from 1 to 200, from 5 to 50, from 10to 20, different combinations of the upper and lower limits describedabove or combinations of any integer in the ranges listed above. TheGaussian filter 2 may comprise a standard deviation from 1 to 200, from5 to 100, from 20 to 60, different combinations of the upper and lowerlimits described above or combinations of any integer in the rangeslisted above. Non-limiting examples of combinations of Gaussian filter 1and Gaussian filter 2 of the DoG filter are described in Table 4 below.

TABLE 4 Gaussian Filter 1 Gaussian Filter 2 Standard Deviation StandardDeviation  1 to 200  1 to 200  5 to 50  5 to 100 10 to 20 20 to 60

The output device 12 b may include but is not limited to a touch screendisplay, a non-touch screen display, a printer, a projector forprojecting the plurality of tiles each having uniquely assigned singledegree of indicium on a display surface such as for example a mirror asdescribed hereinafter with respect to FIG. 2.

FIG. 2 is a perspective view of the system 10 configured as astand-alone imaging system that is located at a retail cosmetics counterfor the purpose of visualizing at least one cosmetic skin attribute andrecommending cosmetic and skin care products based on the visualized atleast one cosmetic skin attribute. FIG. 3 is a block diagram of thesystem 10 of FIG. 2. Referring to FIGS. 2 and 3, the system 10 comprisesa housing 11 for the apparatus 14 of FIG. 1 connected to an imageobtaining device 18 for acquiring a digital image of a subject forvisualizing at least one cosmetic skin attribute. Referring to FIG. 2,the system 10 may comprise a mirror 16, and the image obtaining device18 may be mounted behind the mirror 16 within the housing 11 so that theimage obtaining device 18 may be hidden from view. The image obtainingdevice 18 may be a digital camera, an analog camera connected to adigitizing circuit, a scanner, a video camera or the like. The system 10may include lights 30 such as LED lights arranged about the housing 11to form an LED lighting system for assisting in generating a digitalimage of a subject. The system 10 has an input device 112 a forreceiving a user input. The system 10 may further comprise an outputdevice 112 b such as a projector configured to receive and project thefacial map 30 for display on the mirror 16. The projector is not shownin FIG. 2 as it may be a peripheral component that is separate from thehousing 11 but coupled to the apparatus 14 to form the system 10. Thesystem 10 may further comprise a second output device 112 c such as oneor more speakers optionally coupled to an amplifier for generating audioguidance output to complement and/or enhance an overall consumerexperience.

Preferably obtaining at least one color channel may comprise obtainingat least two color channels, more preferably three color channels. Inparticular, the red color channel, the yellow color channel and the bluecolor channel may be described as follows. When the red color channel isin the L*a*b* color system, a-entropy is an entropy value of thefiltered red color channel. When the yellow color channel is in theL*a*b* color system, b-entropy is an entropy value of the filteredyellow color channel. When the blue color channel corresponds to atexture channel, c-entropy is an entropy value of the blue colorchannel.

The method 200 may further comprise a step of comparing at least onecosmetic attribute to a pre-defined dataset to assign an index. Theindex may be described as a Skin Attribute Index of the analyzedvisually imperceivable consumer skin attribute, wherein the assigning asingle degree of indicium uniquely to each tile is based on the assignedindex. The plurality of tiles each having a uniquely assigned index maybe displayed in a further step after the step of comparing.

Method of Visualizing Entropy Statistics

The present invention also relates to a method of visualizing entropystatistics or entropy values of at least one cosmetic skin attribute ofat least a portion of a face of a subject in a digital image. The methodis described with reference to FIGS. 8A and 8B which is a series ofprocess flow diagrams illustrating how the entropy values arevisualized, and FIG. 9 is a flow chart of a method 300 of visualizingentropy statistics or entropy values of at least one cosmetic skinattribute of at least a portion of a face of a subject in a digitalimage.

A digital image 51 of at least a portion of the face is illustrated inFIG. 8A. The digital image 51 includes an area of the at least a portionof the face 1 defined by a boundary line 52 and comprises a plurality oftiles 54 across the digital image 51, each of the plurality of tiles 54having at least one cosmetic skin attribute analyzed using entropystatistics. An outer periphery 53 envelopes the boundary line 52surrounding the first digital image 51. The first digital image 51 isformed by a total number of pixels, for example, the first digital image51 may have a number of pixels defining an overall image size of thefirst digital image 51. For example, if the tile size is set at 40 by 40pixels to 70 by 70 pixels, accordingly, the number of tiles 54 that formthe plurality of the tiles 54 across the first digital image 51 will beobtained by dividing the overall image size by the specified tile size.It will be appreciated that a size of the tile 54 may be defined by anumber of pixels on a horizontal side (tile width, W) and a number ofpixels on a vertical side (tile height, H). Each tile may comprise atile size of not greater than 100 by 100 pixels, from 1 by 1 pixels to100 by 100 pixels, from 2 by 2 pixels to 100 by 100 pixels, from 5 by 5pixels to 90 pixels by 90 pixels, from 40 by 40 pixels to 70 by 70pixels or different combinations of the upper and lower limits describedabove or combinations of any integer in the ranges listed above. Atechnical effect of having the tile size in the above ranges is that itenables a shorter processing time for analysis of the image data, andaccordingly enable a display to visualize at least one cosmetic skinattribute in a shorter amount of time.

Referring to FIG. 9, the method 300 comprises receiving the digitalimage 51 in step 302 and in step 304, a single degree of indicium 40 isassigned uniquely to each tile 54 of the defined plurality of tilesbased on the analyzed at least one cosmetic skin attribute. At leastsome of the plurality of tiles, each having uniquely assigned singledegree of indicium are displayed in step 306 to visualize an entropyvalue of at least one cosmetic skin attribute as shown in FIG. 8B.

To explain the way the system 10 and the methods 90, 200, 300 work todetermine and visualize at least one cosmetic skin attribute accordingto the present invention, it is helpful to understand how a digitalimage of a face of the subject is obtained in step 202, how the colorchannel image is extracted from the obtained digital image in step 204,how the extracted at least one color channel image is filtered in step206, how a single degree of indicium is assigned uniquely to each tilein step 304 and how the tiles are displayed in step 306. Accordingly,the steps 202, 204, 206, 208 of the method 200 according to the presentinvention and the steps 302, 304 and 306 of the method 300 are describedhereinafter as individual processes for performing each step. Eachprocess may also be described as a sub-routine, i.e. a sequence ofprogram instructions that performs a corresponding step according to themethods 200, 300 according to the present invention.

Obtaining Digital Image

The step 202 of obtaining digital image according to the method 200according to the present invention is described with reference to FIGS.10A, 10B and 10C which is a series of process flow diagrams illustratinghow digital image data is obtained from the digital image, and FIG. 11is a flow chart of a process 400 of obtaining digital image datacorresponding to the step 202.

An input image 50 a of the face 1 is illustrated in FIG. 10A. The inputimage 50 a may be captured by a user, for example, using the camera 18in a step 402 of the process 400 as shown in FIG. 11. FIG. 10Billustrates a step 404 of cropping the input image 50 a to obtain anedited image data 50 b which comprises at least a portion of the face.The input image 50 a may be cropped by identifying an anchor feature 1 aof the face, including but not limited to facial features such as eyes,nose, nostrils, corners of the mouth or the like, and croppingaccordingly. While the eye is depicted as the anchor feature 1 a asshown in FIG. 10B, it will be appreciated that this is merely an exampleand any prominent or detectable facial feature(s) may be an anchorfeature. The edited image data 50 b may be a first digital image 51 thatis obtained in step 404. Alternatively, as shown in FIG. 10C, the editedimage data 50 b may be further processed by cropping to remove one ormore unwanted portions of the input image 50 a thereby obtaining thefirst digital image data 51 which includes the at least a portion of theface 1 defined by a boundary line 52 in step 408. The obtained firstdigital image 51 may comprise at least one region of interest (ROI) 2 ofthe at least a portion of the face 1 that is defined by the boundaryline 52. The ROI 2 may be the entire portion of the face 1, preferablyat least a portion of the face, more preferably, one or more skinregions that defines the at least portion of the face 1. Details of howthe skin regions are defined are described hereinafter with reference toFIGS. 14A to 14C, and the flowchart of FIG. 15.

Optionally, the process 400 may comprise step 406 in which the ROI 2 maybe selected from the group consisting of: a skin region around the eye(“eye region 2 a”), a skin region around the cheek (“cheek region 2 b”),a skin region around the mouth (“mouth region 2 c”), and combinationsthereof, preferably the ROI 2 is a part of the at least a portion of theface 1 of the subject, more preferably the obtained first digital imagedata define a left or right side of the face 1. The ROI 2 may comprisean area of at least 5%, from 10% to 100%, from 25% to 90% of theobtained first digital image.

Defining Tiles

FIG. 12 is a picture illustrating a plurality of tiles 54 on the firstdigital image data 51. FIG. 13 is a flow chart illustrating a process500 of defining the plurality of tiles 54 on the first digital imagedata 51. Referring to FIG. 12, the first digital image 51 includes theat least a portion of the face 1 defined by a boundary line 52 asdescribed hereinbefore with reference to FIG. 10C. Referring to FIG. 13,the process 500 comprises defining an outer periphery 53 enveloping theboundary line 52 surrounding the obtained first digital image (step502). The obtained first digital image 51 is formed by a total number ofpixels, for example, the obtained first digital image 51 may have anumber of pixels which is determined at step 404 or step 406 dependingan image size after cropping of the input image 50 a. Accordingly, anoverall image size based on the obtained first digital image 51 may bedefined in step 504. For example, if the tile size is set at 40 by 40pixels to 70 by 70 pixels, accordingly, the number of tiles 54 that formthe plurality of the tiles 54 across the obtained first digital image 51in step 506 will be obtained by dividing the overall image size by thespecified tile size.

Displaying

The methods according to the present invention may further comprise astep of generating an image description corresponding to the generatedSkin Attribute Index described hereinbefore for visualizing a cosmeticskin condition. The image description may comprise a heat map (such asshown in FIG. 8B, FIG. 21), an aggregate score (such as skin age shownin the fourth area 194 in FIG. 19D, feature 934 in FIG. 22), andcombinations thereof. The aggregate score may be computed based on thegenerated Skin Attribute Index described hereinbefore.

FIGS. 14A to 14C are process flow diagrams illustrating details of aprocess of displaying a plurality of tiles according to step 306 of themethod 300 of the present invention. FIG. 15 is a flow chartillustrating a process 600 of displaying the plurality of tiles. FIG.14A is a picture illustrating a second digital image 60 interposed onthe first digital image data 51. The second digital image 60 includes atleast a portion of the face of the subject with displayed plurality oftiles 54 each having uniquely assigned single degree of indicium 40.FIG. 14B illustrates three zones, a first zone 110, a second zone 120, athird zone 130 displayed on the obtained first digital image based onthe plurality of tiles 54 each having uniquely assigned single degree ofindicium. Each zone 110, 120, 130 identifies a respective region ofinterest (ROI) 2 on the face 1 of the subject described hereinbeforewith reference to FIGS. 9A to 9C and FIG. 10. FIG. 14C differs from FIG.14B in that a boundary line 52 and an outer periphery 53 is displayed inthe second digital image data 60 of FIG. 14B but are not displayed inthe second digital image 60 of FIG. 14C. The first zone 110 may comprisea first zone line having a first zone color 110 a, the second zone 120may comprise a second zone line having a second zone color 120 a and thethird zone 130 may comprise a third zone line having a third zone color130 a. Based on the analyzed image data of the tiles 54 in each zone, acolor of each zone lines may be different to better visually distinguishthe tiles that visualize cosmetic skin attributes which may be in anormal, beautiful, or vulnerable condition relative to the other zonesof the subject, such as for example as illustrated in an exemplary userinterface of FIG. 19D.

FIG. 15 is a flow chart illustrating a process 600 of displaying theplurality of tiles in step 306 of the method 300 according to thepresent invention. The process 600 may begin in step 602 in which theprocessor reads analyzed image data of each tile 54 and assigns a singledegree of indicum uniquely to each tile 54 of the plurality of tilesbased on the analyzed at least one visually cosmetic skin attribute ofthe tile 54 (step 604). When the single degree of indicium isillumination, the analyzed image data of each of the tiles may beconverted to reflect a corresponding degree of brightness of theillumination at each tile in step 606. In an exemplary example, the zone110 may have a lower degree of illumination at each of the tiles withinthe zone 110 relative to a degree of illumination at each of the tileswithin the zone 120. Further, in step 608, the zones may be defined suchthat the first zone 110 may correspond to an eye zone, the second zone120 may correspond to a cheek zone and the third zone 130 corresponds toa mouth zone. An average index of the tiles of each zone may becalculated to generate a diagnosis of a skin condition correlating to adisplayed cosmetic skin attribute according to the respective zone so asto assign a product recommendation item to the zone for treating thedisplayed cosmetic skin attribute in the zone. Specifically, the method300 may further comprise displaying at least one product recommendationitem to treat the displayed cosmetic skin attribute.

FIGS. 16A to 16D are process flow diagrams illustrating a method ofvisualizing at least one cosmetic skin attribute according to thepresent invention. FIG. 17 is a flow chart illustrating a method 700 ofvisualizing at least one cosmetic skin attribute according to thepresent invention. FIG. 16A is a color picture illustrating a firstdigital image of at least a portion of a face of a subject that isdisplayed in step 702 of the method 700 of FIG. 17. FIG. 16B is a colorpicture illustrating a second digital image of at least a portion of aface of a subject and a plurality of tiles each having uniquely assignedsingle degree of indicium, wherein the second digital image isinterposed on the first digital image in step 704. Optionally, the firstdigital image may be converted into grey scale as shown in FIG. 16C toprovide better contrast between the plurality of tiles each havinguniquely assigned single degree of indicium and the first digital image.In step 706, three zones are displayed on the second digital image basedon the plurality of tiles each having uniquely assigned single degree ofindicium.

FIG. 18 is a flow chart illustrating a variation of the method 700 ofvisualizing at least one cosmetic skin attribute as illustrated in FIG.17. At least one product recommendation item is displayed in step 708following step 706 of the method 700 of FIG. 17. In step 710, the useris prompted to select to end the method 700 and the method 700terminates in step 712 if the user selects YES. If the user selects NO,steps of the method 200 of FIG. 6 is performed and the method 700returns to step 708.

Human Machine User Interface

The present invention also relates to a human machine user interface(hereinafter “user interface”) for providing a product recommendation totreat at least one cosmetic skin attribute. The user interface may be agraphical user interface on a portable electronic apparatus including atouch screen display/display with an input device and an image obtainingdevice. The user interface may comprise a first area of the touch screendisplay displaying a first digital image of at least a portion of a faceof the subject obtained from the image obtaining device and a seconddigital image interposed on the first digital image, the second digitalimage having the at least a portion of a face of the subject and saiddisplayed plurality of tiles each having uniquely assigned single degreeof indicium. The user interface may further comprise a second area ofthe touch screen display different from the first area, the second areadisplaying a selectable icon for receiving a user input, wherein animage of at least one product recommendation item to treat the displayedcosmetic skin attribute is displayed on the touch screen display if theuser activates the selectable icon.

FIGS. 19A to 19E are screen shots, each illustrating an exemplary userinterface cooperating with each other for visualizing a cosmetic skinattribute according to the present invention. Although FIGS. 19A to 19Eare described as a series of user interfaces which are provided in asequential manner in response to a preceding user interface, it will beappreciated that the user interfaces of FIGS. 19A to 19E may beprogrammed in multiple ways to define an overall user interface forvisualizing at least one cosmetic skin attribute according to methodsaccording to the present invention as described hereinbefore.Preferably, all the user interfaces of FIGS. 19A to 19E define anexemplary user interface for visualizing a cosmetic skin attributeaccording to the present invention.

FIG. 19A depicts a user interface 160 for receiving a first user input,preferably the first user input is the age of the user. The userinterface 160 may comprise a first area 162 for receiving the first userinput. The first area 162 may include one or more user input features164 for receiving the first user input. The user input feature 164 maybe such as for example, a selectable input icon corresponding to apredetermined user feature such as for example a user's age as shown inFIG. 19A. The user interface 160 may further comprise a second area 166including corresponding instructions to the user for providing the firstuser input. The second area 166 may be disposed above the first area 162so as to be provide a more user-friendly interface. The user interface160 may be part of a start option for beginning a method 200 accordingto the present invention.

FIG. 19B depicts a user interface 170 for receiving a second user input,preferably the second user input is a cosmetic skin attribute that iscausing concern to the user. The cosmetic skin attribute may bedescribed as a skin concern of the user. The user interface 170 may beprovided in response to the selection of a first user input from theuser input feature 164 of FIG. 19A. The user interface 170 may comprisea first area 172 for receiving the second user input. The first area 172may include one or more user input features 174 for receiving the seconduser input. The user input feature 174 may be such as for example, aselectable input icon corresponding to a predetermined skin concern. Thefirst area 172 may further comprise an explanatory area 173corresponding to the one or more input features 174 in which theexplanatory area 173 includes a brief description of a cosmetic skinattribute or the skin concern. The user interface 170 may furthercomprise a second area 176 including corresponding instructions to theuser for providing the user input. The second area 176 may be disposedabove the first area 172 so as to be provide a more user-friendlyinterface.

FIG. 19C depicts a user interface 180 for obtaining an input image of auser. The user interface 180 may comprise a first area 182 withinstructions for aligning an anchor feature (such as eyes) so as toobtain the first digital image data according to the process 400 asdescribed in FIG. 11. The user interface 180 may be provided in responseto the selection of the second user input through the one or more userinput features 174 of FIG. 19B.

FIG. 19D depicts a user interface 190 for displaying at least onecosmetic skin attribute. The user interface 190 may be provided afterthe input image of the user is obtained in the user interface 180 ofFIG. 19C. As shown in FIG. 19D, the at least one portion of skin of theperson is facial skin; wherein the facial skin comprises at least oneregion of interest (ROI), which is preferably selected from the groupconsisting of cheek region/zone, eye region/zone, forehead region/zone,nose region/zone, and combinations thereof; and wherein the imagedescription visualizes a need for improvement in said at least one ROIor a difference in the cosmetic skin attribute between a first ROI and asecond ROI.

The user interface 190 may comprise a first area 191 displaying theplurality of tiles each having uniquely assigned single degree ofindicium to visualize at least one cosmetic skin attribute according tomethods of the present invention. The first area 191 may display similarfeatures as shown in FIG. 18D but differs only in that lines definingthe plurality of tiles may be turned off and/or set as an invisiblelayer. The first area 191 may comprise a first zone 110 corresponding toan eye zone of the at least a portion of the face of the user, a secondzone 120 corresponding to a cheek zone of the at least a portion of theface of the user, and a third zone 130 corresponding to a mouth zone ofthe at least a portion of the face of the user. As shown in FIG. 19D, azone result may be displayed in a third area 193 whereby the zone resultcomprises an index which may be generated for each zone based on arelative comparison of the indexes of the zones within the at least aportion of the face of the user. In an exemplary embodiment, dependingon the zone results, the first zone 110 may be described as anormal/beautiful/vulnerable zone, the second zone 120 may be describedas a normal/beautiful/vulnerable zone and the third zone 130 may bedescribed as a normal/beautiful/vulnerable zone. Preferably, each zonemay have different descriptions based on the relative differences in thezone results. The user interface 190 also includes a second area 192 forreceiving a third user input. The second area 192 may include one ormore user input features 1921 for receiving the third user input. Theuser input feature 1921 may be such as for example, a selectable inputicon for proceeding with a next step of the method according to thepresent invention. Optionally, the user interface 190 may comprise afourth area 194 for displaying a skin age of the user based on theanalyzed at least one cosmetic skin attribute of each tile of theplurality of tiles based on the obtained first digital image data of theat least a portion of the face of the user.

FIG. 19E depicts a user interface 800 comprising a first area 801 fordisplaying a product recommendation item 210. The user interface 800 maybe provided in response to selection of the user input feature 1921 fromthe user interface 190 of FIG. 19D. Optionally, the user interface 800may comprise a second area 802 for providing details of the productrecommendation item 210. Preferably, the user interface 800 may comprisea third area 803 for receiving a fourth user input such as for examplerequest for assistance from a product consultant for enquiry and/orpurchase of the product recommendation item 210. The third area 803 mayinclude one or more user input features 2031 for receiving the fourthuser input. The user input feature 2031 may be such as for example, aselectable input icon for proceeding with a next step of the methodaccording to the present invention.

FIG. 20 depicts a partial view of an exemplary user interface 900comprising an image description 901 overlaid on a digital image 51 forvisualizing at least one cosmetic skin attribute according to thepresent invention. The image description 902 comprises a heat mapgenerated based on the entropy values output from the method 90described hereinbefore. The heat map comprises a first heat map section906 based on low entropy values which correspond to a better cosmeticskin attribute condition. The heat map 904 further comprises a secondheat map section 908 based on high entropy values correspond to a poorercosmetic skin attribute condition. The first heat map section 906 isformed of a first plurality of tiles which is visually different from asecond plurality of tiles in the second heat map section 908. Forexample, the first plurality of tiles is converted to display adifferent color from the color of the second plurality of tiles. Heatmap sections 910 which are not displayed (hereinafter “non-displayedheat map sections 910”) correspond to entropy values between the highand low entropy values. The heat map sections may be configured asfollows to display entropy information related to the cosmetic skinattribute condition and the Skin Attribute Index as outlined in Table 5below.

TABLE 5 Heat Map Section Entropy Cosmetic Skin Heat Map VisualizationValues Attribute Condition Heat Map Displayed as first color Low BetterSection 906 Heat Map Displayed as second color High Poor Section 908different from first color Heat Map Not Displayed Between Low BetweenPoor and Section 910 and High Better

FIG. 21 depicts an alternate variation of an image description 920 forvisualizing at least one cosmetic skin attribute in the user interface900 of FIG. 19. The image description 920 differs from the imagedescription 902 of FIG. 19 in that the image description 920 comprises adisplayed region of interest (ROI) 922 wherein the displayed ROI 922 isconverted to display a color to indicate poorer cosmetic skin attributecondition relative to other non-displayed regions of interest (ROI) 924which correspond to better cosmetic skin attribute condition. Anadvantage of only displaying a single heat map section (see FIG. 20) orROI is that the consumer viewing the user interface is not overloadedwith too much visual information.

FIG. 22 is a screen shot illustrating an exemplary user interface 930for visualizing at least one cosmetic skin attribute according to thepresent invention, wherein the at least one cosmetic skin attribute isskin purity. The user interface 930 differs from the user interface 902of FIG. 19 in that the user interface 930 comprises alternate text 932describing the cosmetic skin attribute and an aggregate score 934 basedon the generated Skin Attribute Index. The user interface 930 mayfurther comprise a meter 936 and a meter marker 938 for representing theaggregate score on a scale of 0 to 100 along the meter 936. The meter936 is a different way of visualizing the aggregate score 934, and maybe optional. A color of the meter 936 may be configured to show agradient of colors representative of the first heat map section 904 andthe second heat map section 906.

The methods for determining a cosmetic skin condition according thepresent invention described hereinbefore may further comprise a step oftracking the cosmetic skin attribute over a predetermined period oftime. For example, the user interface 930 as shown in FIG. 21 maycomprise a first selectable icon 940 which upon selection, causesinstructions to be received by and steps performed by the processor togenerate a calendar or schedule to create a cosmetic skin attributediary to track improvement of cosmetic skin attributes. For example,when the consumer uses it on Day 1, the date and facial analysis isrecorded and saved in the memory. Subsequently, whenever the consumeruses the method according to the present invention in future (after apredetermined period, 1 week, 1 month, 6 months), the facial skin of theconsumer is analyzed again and the consumer can compare how his/herfacial skin looks at the time after the predetermined period relative toDay 1. The methods according to the present invention may be configuredto be a downloadable software application that is stored as a nativeapplication on a portable electronic device or a web application thatcan be accessed through a login account specific to a consumer, so thatthe consumer can perform a self-skin analysis based on the methodsaccording to the present invention and view and/or monitor theimprovement (reduction in the ROIs with poorer cosmetic skin attributecondition) over a period of time.

The user interface 930 may further comprise a second selectable icon 942which upon selection, enables the method for determining a cosmetic skinattribute according to the present invention to be repeated. Forexample, the method 90 described hereinbefore may be repeated.

The dimensions and values disclosed herein are not to be understood asbeing strictly limited to the exact numerical values recited. Instead,unless otherwise specified, each such dimension is intended to mean boththe recited value and a functionally equivalent range surrounding thatvalue. For example, a dimension disclosed as “40 mm” is intended to mean“about 40 mm.”

Every document cited herein, including any cross referenced or relatedpatent or application and any patent application or patent to which thisapplication claims priority or benefit thereof, is hereby incorporatedherein by reference in its entirety unless expressly excluded orotherwise limited. The citation of any document is not an admission thatit is prior art with respect to any invention disclosed or claimedherein or that it alone, or in any combination with any other referenceor references, teaches, suggests or discloses any such invention.Further, to the extent that any meaning or definition of a term in thisdocument conflicts with any meaning or definition of the same term in adocument incorporated by reference, the meaning or definition assignedto that term in this document shall govern.

While particular embodiments of the present invention have beenillustrated and described, it would be obvious to those skilled in theart that various other changes and modifications can be made withoutdeparting from the spirit and scope of the invention. It is thereforeintended to cover in the appended claims all such changes andmodifications that are within the scope of this invention.

What is claimed is:
 1. A method of determining a cosmetic skin attributeof a person, the method comprising: a) obtaining at least one colorchannel image comprising at least one portion of skin of the person; b)analyzing the at least one color channel image using entropy statisticsto obtain an entropy value; and c) determining the cosmetic skinattribute of the at least one portion of skin of the person based on theentropy value; wherein a Skin Attribute Index is generated as aprobability value indicative of a condition of the cosmetic skinattribute of the at least one portion of skin of the person relative toa defined population of people; wherein the Skin Attribute Index isgenerated as a function of the entropy value defined by F(EntropyValue), wherein said function is determined by a model established upona training dataset wherein the training dataset comprises (i) aplurality of color channel images of a the defined population of people,wherein each of the plurality of color channel images comprises facialskin of a person in the defined population of people, wherein the facialskin comprises the cosmetic skin attribute, and (ii) an associated classdefinition based on the cosmetic skin attribute; wherein the at leastone color channel image is an image in the L*a*b* color system selectedfrom the group consisting of an L-channel image, an a-channel image, ab-channel image and a c-channel image from RGB color system, andcombinations thereof; wherein the entropy value is selected from thegroup consisting of a L-entropy value, an a-entropy value, a b-entropyvalue, a c-entropy value, and combinations thereof; and wherein thefunction has the formula: Skin Attribute Index=A+B×(L-entropyvalue)+C×(a-entropy value)+D×(b-entropy value)+E×(c-entropy), wherein A,B, C, D, and E are constants; wherein at least one of B, C, D, and E isnot
 0. 2. The method according to claim 1, wherein the at least onecolor channel image is an L-channel image; wherein the entropy value isan L-entropy value; wherein C, D, and E each have a value of 0; andwherein the generated Skin Attribute Index is indicative of skin purity,skin tone or skin radiance.
 3. The method according to claim 1, whereinthe at least one color channel image is an a-channel image, the entropyvalue is an a-entropy value, and B, D, and E each have a value of 0, andwherein the generated Skin Attribute Index is indicative of skininflammation.
 4. The method according to claim 1, wherein the at leastone color channel image is a b-channel image, the entropy value is ab-entropy value, and B, C, and E each have a value of 0, and wherein thegenerated Skin Attribute Index is indicative of skin pigmentation orskin dullness.
 5. The method according to claim 1, wherein the at leastone color channel image is a c-channel image, the entropy value is ac-entropy value, and B, C, and D each have a value of 0, and wherein thegenerated Skin Attribute Index is indicative of skin topography.
 6. Themethod of claim 5, wherein the skin topography is selected from thegroup consisting of pores, fine lines, wrinkles, sagging, skinelasticity, and combinations thereof.
 7. The method of claim 1, furthercomprising generating and displaying an image description correspondingto the generated Skin Attribute Index.
 8. The method of claim 1, whereinthe model is a regression model or a classification model.
 9. The methodof claim 8, wherein said regression model is a regression model selectedfrom the group consisting of a linear regression model, a machinelearning linear regression model, a machine learning support vectorregression model, or a random forest regression model.
 10. The methodaccording to claim 1, wherein the cosmetic skin attribute is selectedfrom the group consisting of skin purity, skin age, skin topography,skin tone, skin pigmentation, skin pores, skin inflammation, skinhydration, skin sebum level, acne, moles, skin radiance, skin shine,skin dullness, and skin barrier.
 11. The method according to claim 10,wherein the cosmetic skin attribute is a visually imperceivable cosmeticskin attribute, wherein the visually imperceivable cosmetic skinattribute is a cosmetic skin attribute which is not detectable by anunaided eye, or a cosmetic skin attribute detectable visually by aconsumer but the consumer does not understand the cosmetic skinattribute.
 12. The method of claim 1, wherein prior to analyzing, the atleast one color channel image is filtered by using a frequency filterselected from the group consisting of Fourier Transformation Filter,Wavelet Transformation, and Difference of Gaussian (DoG) filter.
 13. Themethod of claim 12, wherein the first Gaussian Filter has a standarddeviation of 5 to 50 and the second Gaussian filter has a standarddeviation of 5 to
 100. 14. The method of claim 1, wherein the at leastone portion of skin of the person is facial skin; wherein the facialskin comprises at least one region of interest (ROI) selected from thegroup consisting of cheek region, eye region, forehead region, noseregion, and combinations thereof, and wherein the image descriptiondisplays a need for improvement in said at least one ROI or a differencein the cosmetic skin attribute between a first ROI and a second ROI.